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DOCUNENT- IDENTIFIER: US 6034693 A 

TITLE: Image synthesizing apparatus, image synthesizing method and information 
storage medium 

Abstract Text (1) : 

An image synthesizing apparatus, an image synthesizing method and an information 
storage medium which can realize the representation of the front and back sides of 
a primitive surface while decreasing processing load. Whether the front or back 
side of a polygon (primitive surface) is to be displayed on a screen is judged. 
When the front side of the polygon is to be displayed, a first texture is mapped 
(on the polygon) . On the other hand, if the back side of the polygon is to be 
displayed, a second texture is mapped (on the polygon) . Thus, a realistic 
representation can be accomplished without increasing processing load. Depending on 
whether the front or back side of the polygon is to be displayed, various image 
information such as color palette, vertex brightness information, normal vector 
information lighting model parameter information and transparency information may 
be varied. 

Brief Summary Text (5) : 

There are known many types of image synthesizing systems in which an object to be 
displayed is formed by polygons (primitive surfaces), such objects being used to 
synthesize an image. For example, in a three-dimensional racing game system, 
polygons are used to form other racing cars, guard-rails, signboards, roads, etc. 
to be displayed on a screen. A player operates its racing car using given operating 
means (steering wheel, accelerator pedal, game controller and others) for enjoying 
a car race with the other racing cars. In such an image synthesizing system, the 
viewpoint and sight line of the player vary depending on the operation of the 
player. This causes such a situation that the back side of the polygons will be 
displayed on the screen. Assuming a case that a signboard formed by polygons is to 
be displayed on the screen, the front side of the polygons is only displayed on the 
screen when a racing car on the screen goes forward. If the racing car on the 
screen goes backward, the back side of the polygons will be displayed. 

Brief Summary Text (7) : 

On the other hand, another image synthesizing system of such a type that faces of a 
pair of polygons each used to form the" back and front side of a signboard to be 
displayed are put together, raises another problem in that the number of polygons 
used to form an object is duplicated. Such an image synthesizing system is required 
to provide a real-time operation in which an image is synthesized in real time for 
the player's operation. When the number of polygons to be processed is increased, 
the processing load is correspondingly increased. Thus, the real-time operation 
will not be maintained, resulting in deterioration of the image quality. 

Brief Summary Text (9) : 

In view of the aforementioned technical problems, the present invention is made and 
has its object to provide an image synthesizing apparatus, an image synthesizing 
method and an information storage medium which can be used to realize the display 
of front and back sides of primitive surfaces without increasing processing load so 
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much . 

Brief Summary Text (10) : 

To this end, the present invention provides an image synthesizing apparatus for 
generating an image including oblects each of which is formed by at least one 
primitive surface comprising: 

Brief Summary Text (14) : 

The present invention also provides an image synthesizing apparatus for generating 
an image including objects each of which is formed by at least one primitive 
surface comprising: 

Brief Summary Text (16) : 

means for forming an image of the primitive surface by referring to a first color 
palette through a color code when the front side thereof is to be displayed and for 
forming an image of the primitive surface by referring to a second color palette 
through a color code when the back side thereof is to be displayed. 

Brief Summary Text (17) : 

According to the present invention, different color palettes are referred to form 
different images for the front and back sides of the primitive surface . Thus, a 
color which cannot be used to display the front side of the primitive surface may 
be used to display the back side of the primitive surface . 

Brief Summary Text (18) : 

The present invention further provides an image synthesizing apparatus for 
generating an image including objects each of which is formed by at least one 
primitive surface comprising: 

Brief Summary Text (20) : 

means for computing the brightness of the primitive surface by using at least one 
of vertex brightness information, normal vector information and lighting model 
parameter information and for varying the brightness information, the normal vector 
information and the lighting model parameter information depending on whether the 
front or back side of the primitive surface is to be displayed. 

Brief Summary Text (22) : 

The present invention further provides an image synthesizing apparatus for 
generating an image including objects each of which is formed by at least one 
primitive surface comprising: 

Brief Summary Text (24) : 

means for varying image information used to form an image of the primitive surface 
depending on whether the front or back side of the primitive surface is to be 
displayed. Thus, the front and back sides of an object can be represented with 
reality and without increasing the number of a primitive surface to be used. In 
such a case, it is desirable that the image information may be at least one of 
texture information, information for determining texture, color palette 
information, information for determining color palette, brightness information, 
information for determining brightness, transparency information and information 
for determining transparency. 

Detailed Description Text (4) : 

FIG. 1 shows a view illustrating the principle of a first embodiment of the present 
invention. An object to be displayed 10 is formed by a plurality of polygons. 
Letters U and V represent coordinates in a texture plane. In this embodiment, for 
example, it is first judged whether the front or back side of a polygon 2 0 is to be 
displayed on a screen. If it is judged that the front side of the polygon is to be 
displayed, a first texture (texture for the front side) is mapped to the polygon 
20. On the other hand, if it is judged that the back side of the polygon is to be 
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displayed, a second texture (texture for the back side) is mapped to the polygon 
20. Thus, the front and back sides of the polygon can be realized by a single 
polygon without increasing processing load. On the contrary, in a system which the 
back side of a polygon cannot be displayed, the polygon will instantly vanishes 
when the polygon is reversed. It does not provide a sense of reality. In a system 
which faces of two polygons are put together to form both the front and back sides 
surface of the object, two polygons are required to form one surface with increase 
of the number of polygons to be processed, resulting in increase of processing 
load. On the contrary, this embodiment of the present invention can realize the 
realistic representation of the front and back surface of an object without 
increasing processing load. 

Detailed Description Text (6) : 

Various techniques for judging the front and back sides of the polygon may be taken 
in the present invention. In a first technique, as shown in FIG. 3A, vertex numbers 
have been previously provided to the respective vertices of a polygon. The 
displayed face of the polygon is judged to be front or back side depending on 
whether these vertex numbers are sequentially followed anti -clockwise or clockwise. 
A second technique computes the vector product of the vertex coordinates in a 
polygon. The vector product is determined in respect to its plus or minus Z- 
coordinate. The result is used to judge the front or back side of the polygon. This 
corresponds to determination of the direction of the normal vector in the polygon 
surface. A third technique is such that if a moving body 22 manipulated by a player 
(whose viewpoint follows the moving body) is outside of a dome 24, it is judged 
that the polygon forming the dome 24 is the front side. Thus, the front side of the 
polygon is displayed. If the moving body 22 is inside of the dome 24, it is then 
judged that the polygon to be displayed is the back side. Thus, the back side of 
the polygon is displayed. In such a manner, a single polygon is only used to 
represent the dome 24 without putting faces of two polygons together. 

Detailed Description Text (7) : 

The judgment of the front and back sides of the polygon is not limited to the 
aforementioned three techniques, but may be carried out through any of various 
other techniques. Although it is generally desirable that the texture information 
is color information such as color codes or the like, any other information such as 
translucent information, brightness information, surface shape information (e.g., 
normal vector information or displacement information) , reflectance information, 
index of refraction information, depth information or the like can be applied. 

Detailed Description Text (12) : 

FIG. 4 shows a view illustrating the principle of a second embodiment of the 
present invention. As in the first embodiment, the second embodiment judges whether 
the front or back side of a polygon is to be displayed on a screen. If it is judged 
that the front side of the polygon is to be displayed, a first color palette is 
referred to through a color code to form image information. More particularly, RGB 
information to be displayed at objective dots is determined. On the other hand, if 
it is judged that the back side of the polygon is to be displayed, a second color 
palette is referred to through a color code to form image information. In such a 
manner, different color palettes can be used for displaying back and front side of 
the polygon to be displayed. Thus, for example, the front side of the polygon may 
be more brightened while the back side of the polygon may be more darkened. 
Furthermore, colors that are represented by a combination of RGB but cannot be used 
on representation of the front side of the polygon can be used on representation of 
the back side of the same object . In FIG. 3B, for example, a polygon forming a dome 
24 has the same color code both for the front and back sides thereof, but only a 
color palette is changed. More particularly, when a moving body 22 is to be moved 
into the dome 24 and if the back side of the polygon forming the dome 24 is to be 
displayed, a color palette for representing a more darkened color will be selected. 
Thus, the dome 24 having a specific color on its outer side may be represented with 
the more darkened color than the color specified for the outer side when the moving 
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body has entered into the dome 24. This provides a realistic representation without 
increasing processing load so much. 

Detailed Description Text (17) : 

The third embodiment is such that depending on whether the front or back side of 
the polygon is to be displayed, at least one of vertex brightness information, 
normal vector information and lighting model parameter information all of which are 
used to compute the brightness of a polygon is varied. 

Detailed Description Text (18) : 

With Gouraud shading, for example as shown in FIG. 6A, vertex brightness 
information laO, lal, Ia2 and Ia3 for displaying the front side of a polygon is 
different from vertex brightness information IbO, Ibl, Ib2 and Ib3 for displaying 
the back side of a polygon. Thus, the brightness and shading of the polygon can be 
varied depending on whether the front or back side of the polygon is to be 
displayed. This can be accomplished with realistic representation without 
increasing processing load so much. Technique for varying the vertex brightness 
information may be provided by causing each polygon to have two types of vertex 
brightness information laO to Ia3 and IbO to Ib3 as one of polygon information. It 
may also be provided by causing vertex brightness information ID to 13 as polygon 
information to be subjected to various computations, thereby obtaining the front 
and back side vertex brightness information laO to Ia3 and IbO to Ib4 , as shown in 
FIG. 6B. For example, addition of a predetermined value to 10 to 13 may generate 
laO to Ia3 while subtraction of a predetermined value from 10 to 13 may generate 
IbO to Ib3 . In such a case, the whole object may be brightened if the front side of 
the polygon is displayed while the whole object may be darkened if the back side 
thereof is displayed. 

Detailed Description Text (19) : 

With Phong's smooth shading, for example, as shown in FIG. 6C, normal vector 
information naO, nal, na2 and na3 (or nac) for displaying the front side of a 
polygon is different from normal vector information nbO, nbl, nb2 and nb3 (or nbc) 
for displaying the back side of the polygon. This can also vary the brightness or 
shading of the polygon depending on whether the front or back side of the polygon 
is to be displayed. This can also be accomplished with realistic representation and 
without increasing processing load so much. Technique for varying the normal vector 
information may be provided by causing each polygon to have two types of normal 
vector information naO to na3 (or nac) or nbO to nb3 (or nbc) as polygon 
information. It may also be provided by causing normal vector information nO to n3 
(or nc) as polygon information to be subjected to various computations, thereby 
obtaining normal vector information naO to na3 (or nac) and nbO to nb3 (or nbc) for 
the front and back sides of the polygon, as shown in FIG. 6D. 

Detailed Description Text (20) : 

When vertex brightness information is determined from normal vector information and 
if this vertex brightness information is used to perform the Gouraud shading 
process, the normal vector information may be varied depending on whether the front 
or back side of the polygon is to be displayed. 

Detailed Description Text (21) : 

In place of or together with the vertex brightness information and normal vector 
information, lighting model parameter information may be varied. 

Detailed Description Text (26) : 

In such a case, the amount of light incident on the surface of an material per unit 
area is proportional to COS .theta. if an angle included between a normal vector N 
and a light vector L on a plane of FIG. 7A is .theta.. If COS .theta. is minus, the 
amount of light is zero. If the length of the vectors N and L are equal to 1, 
cos .theta. can be represented by the dot product of the vector N and L, or 
N.multidot . L . This dot product N.multidot.L may be used to determine the 
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coefficient d of the diffuse component as follows: 
Detailed Description Text (28) : 

Phong shading which uses a lighting model also using the specular component of the 
light will now be described. The specular reflection is observed on a surface. 
Since the surface of an material is rough, the specular reflection has a spread 
direction. The intensity of the specular reflection increases as an angle .beta, 
(^ri angle represented between vectors N and H) of FIG. 7B decreases. 

Detailed Description Text (29) : 

H is equal to (E+L) / . vertline . E+L . vertline . and is a vector of which direction is 
half way between the vectors E and L. Since the specular reflection is generated at 
a portion of the material surface oriented toward the direction H, the intensity of 
the specular reflection depends on the dot product of the vector H and N. With 
Phong shading, the coefficients of the diffuse and specular components are 
represented as follows: 

Detailed Description Text (30) : 

In the above formulae, it is premised that the length of all the vectors N, L and H 
is equal to 1 . In the lower formula, by varying the value n, various ways of 
spreading the highlight can be obtained. 

Detailed Description Text (32) : 

By varying the parameter information such as specular, ambient and diffuse 
components of the light, light vector and normal vector of the surface in the 
lighting model as described above, depending on whether the front or back side of 
the polygon is to be displayed, a single polygon can be used to display the front 
and back sides of the object with realistic representation. 

Detailed Description Text (34) : 

The information storage medium 1006 mainly stores information such as image and 
sound information for representing objects to be displayed and may take any of 
various media such as CD-ROM, game cassette, IC card, DVD, MO, FD and memory. For 
example, the information storage medium such as CD-ROM, game cassette or DVD is 
used for a domestic game machine to store a game program and the like. With an 
arcade game machine, a memory such as ROM may be used. In such a case, the 
information storage medium 1006 may be the ROM 1002. 

Detailed Description Text (40) : 

FIG. 12A shows an arcade game machine to which the first to third embodiments are 
applied. A player can enjoy a game by manipulating a lever 1102 and buttons 1104 
and others while viewing a game picture displayed on a display 1100. An IC board 
1106, which itself is included in the game machine, includes CPU, image 
synthesizing IC, sound synthesizing IC and others which are mounted therein. A 
memory 1108, which is an information storage medium on the IC board 1106, stores 
information used for such as judging whether the front or back side of the 
primitive surface is to be displayed, and information for varying image information 
used in forming the image of the primitive surface, depending on whether the front 
or back side of the primitive surface is to be displayed. The information will be 
referred to "the stored information". The stored information includes at least one 
of program codes used to perform the aforementioned processings, image information, 
sound information, shape information of objects to be displayed, table data, list 
data, player's information and other information. 

Detailed Description Text (44) : 

In the first to third embodiments of the present invention, the texture for 
mapping, the color palette to be referred to, the brightness information or the 
like is varied depending on whether the front or back side of the primitive surface 
is to be displayed. However, the present invention is not limited to these 
embodiments, but it may be applied to such a configuration at least that the image 
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information used to form the image of a primitive surface is varied depending on 
whether the front or back side of the primitive surface is to be displayed. In this 
case, the image information may include various information such as texture 
information, information for determining texture, color palette information, 
information for determining color palette, brightness information, information for 
determining brightness, transparency information and information for determining 
transparency. 

Detailed Description Text (47) : 

The present invention can synthesize various images. In the racing car game shown 
in FIG. 8A, for example, the signboard polygon 34 may be synthesized into a form as 
viewed from the racing car 3 0 and another form as viewed from the other racing car 
32. As shown in FIGS. 8B and 8C, the front side of the signboard polygon 34 viewed 
from the racing car 3 0 may be displayed on a screen while the back side of the 
signboard polygon 34 viewed from the racing car 32 may be displayed on the screen. 
A Chinese restaurant guided by the signboard must be displayed with its front or 
back side thereof since this shop is in the straight direction from the racing car 
30 and also in the turned direction from the racing car 32. According to the 
present invention, such a change between the front and back sides of the signboard 
in the viewed direction can be accomplished without increasing processing load so 
much since a single polygon is used to realize the front or back side of the 
polygon, as shown in FIGS. SB and SC. More particularly, it is judged that the 
front side of the signboard polygon 34 is to be displayed as it is viewed from the 
racing car 30. Thus, a front side texture is mapped on the sign board and a front 
side color palette is used. The brightness thereof is computed using a front side 
information . On the other hand, if the signboard polygon 34 is viewed from the 
racing car 32, it is judged that the back side of the signboard polygon 34 is to be 
displayed. Thus, a back side texture is mapped on the signboard polygon and a back 
color palette is used. The brightness thereof is also computed using a back side 
information . 

Detailed Description Text (49) : 

Moreover, according to the present invention, the representation of such a water 
surface as shown in FIGS. lOA and lOB can be easily accomplished, FIG. lOA shows an 
image when the water surface is viewed from above, and FIG. lOB shows an image when 
the water surface is viewed from thereunder (or from the underwater) . In such a 
case, it is desirable that the transparency in addition to the texture, color 
palette and brightness are varied. In other words, the transparency is decreased as 
the water surface is viewed from above while the transparency is increased as the 
water surface is viewed from under. Thus, the bottom of the water may be less 
viewed as the water surface is viewed from above. When the water surface is viewed 
from under, a tree located above the water surface may be displayed on the water 
surface. For example, if it is assumed that the color of a transparent (or 
translucent) object is CI and the color of the background is C2 , the whole color 
can be represented by C=tC2+ (1-t ) CI . The transparency may be varied by varying the 
above value t depending on whether the water surface is viewed from above or under. 
In order to provide a distorted image of the tree disturbed by waves, the normal 
vector information used to represent the condition of the waved surface may be 
varied depending on whether the water surface is viewed from above or under. 

Detailed Description Text (50) : 

In addition, the present invention can represent various other representation such 

an object projected onto a mirror or window glass, the color or nerves of a leaf 
different depending on whether the front or back side of that leaf is to be 
displayed, . the internal structure of an object (e.g., the inside of a bonnet in a 
crushed car) , the wings of an airplane, the front and back sides of one page in a 
book, a flag fluttered by the wind and a fan. 

CLAIMS : 
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1. An image synthesizing apparatus for generating an image including objectSi each 
of which is formed by at least one primitive surface, comprising: 

judging means for judging whether the front or back side of said at least one 
primitive surface should be displayed on a screen; and 

mapping means for mapping a first texture to said at least one primitive surface 
when the front side thereof is to be displayed and for mapping a second texture to 
said at least one primitive surface when the back side thereof is to be displayed. 

3. The image synthesizing apparatus of claim 1, wherein the means for judging 
judges whether the front side or the back side of the primitive surface should be 
displayed based on at least one of: 

vertex orientation information that identifies whether vertices of the primitive 
surface are oriented clockwise or counterclockwise, and 

vector product information that identifies a vector product of vertex coordinates 
with respect to a positive or negative z-coordinate system. 

4. An image synthesizing apparatus for generating an image including objects, each 
of which is formed by at least one primitive surface, comprising: 

judging means for judging whether the front or back side of said at least one 
primitive surface should be displayed on a screen; and 

forming means for forming an image of said at least one primitive surface by 
referring to a first color palette through a color code when the front side of said 
at least one primitive surface is to be displayed and for forming an image of said 
at least one primitive surface by referring to a second color palette through a 
color code when the back side of said at least one primitive surface is to be 
displayed. 

6. The image synthesizing apparatus of claim 4, wherein the means for judging 
judges whether the front side or the back side of the primitive surface should be 
displayed based on at least one of: 

vertex orientation information that identifies whether vertices of the primitive 
surface are oriented clockwise or counterclockwise, and 

vector product information that identifies a vector product of vertex coordinates 
with respect to a positive or negative z-coordinate system. 

7. An image synthesizing apparatus for generating an image including objects, each 
of which is formed by at least one primitive surface, comprising: 

judging means for judging whether the front or back side of said at least one 
primitive surface should be displayed on a screen; and 

computing means for computing the brightness of said at least one primitive surface 
by using at least one of vertex brightness information, normal vector information 
and lighting model parameter information and for varying said brightness 
information, said normal vector information and said lighting model parameter 
information depending on whether the front or back side of said at least one 
primitive surface is to be displayed. 

9. The image synthesizing apparatus of claim 7, wherein the means for judging 
judges whether the front side or the back side of the primitive surface should be 
displayed based on at least one of: 
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vertex orientation information that identifies whether vertices of the primitive 
surface are oriented clockwise or counterclockwise, and 

vector product information that identifies a vector product of vertex coordinates 
with respect to a positive or negative z-coordinate system. 

10. An image synthesizing apparatus for generating an image including objects, each 
of which is formed by at least one primitive surface, comprising: 

judging means for judging whether the front or back side of said at least one 
primitive surface should be displayed on a screen; and 

means for varying image information used to form an image of said at least one 
primitive surface depending on whether the front or back side of said at least one 
primitive surface is to be displayed. 

13. The image synthesizing apparatus of claim 10, wherein the means for judging 
judges whether the front side or the back side of the primitive surface should be 
displayed based on at least one of: 

vertex orientation information that identifies whether vertices of the primitive 
surface are oriented clockwise or counterclockwise, and 

vector product information that identifies a vector product of vertex coordinates 
with respect to a positive or negative z-coordinate system. 

14 . An image synthesizing method for generating an image including objects, each of 
which is formed by at least one primitive surface, comprising the steps of: 

judging whether the front or back side of said at least one primitive surface is to 
be displayed on a screen; and 

varying image information used to form the image of said at least one primitive 
surface depending on whether the front or back side of the at least one primitive 
surface is to be displayed. 

18. An information storage medium for generating an image including objects, each 
of which is formed by at least one primitive surface, comprising: 

information used to judge whether the front or back side of the at least one 
primitive surface is to be displayed on a screen; and 

information used to vary image information used to form an image of said at least 
one primitive surface depending on whether the front or back side of the at least 
one primitive surface is to be displayed. 
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